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Abstract 

Outcomes-based program evaluation is a systematic approach to identifying outcome indicators and 
measuring results against those indicators. One dimension of program evaluation is assessing the level of 
learner acquisition to determine if learning objectives were achieved as intended. The purpose of the 
proposed model is to use Bloom’s Taxonomy to guide program and curriculum designers through the 
challenging process of identifying, classifying, and clearly communicating specific outcome indicators by 


which to evaluate their programs. 

Introduction 

Operating in the current era of escalating 
accountability to sponsoring agencies and accrediting 
bodies, accurate program evaluations are becoming 
critical in the increased competition for funding. 
Because resources are limited, greater expectations for 
demonstrated efficiencies and performance outcomes 
are mounting. 

Program Evaluation 

Program evaluations are conducted to assess the 
quality of a program and serve two main functions: to 
evaluate process efficiencies and to evaluate program 
outcomes. Process evaluations focus on the relevancy of 
content and on implementation practices in order to 
identify ways in which to improve upon program 


delivery, services, and administration. Outcomes 
evaluations focus on assessing program results, based 
on participant learning and the impact of this learning 
for stakeholders such as students, funding agencies, and 
the greater community. Outcome measurements “assess 
the extent to which a program has achieved its intended 
results. The main questions addressed are: What has 
changed in the lives of individuals, families, 
organizations or the community as a result of this 
program? Has this program made a difference? How are 
the lives of program participants better as a result of the 
program?” (The Evaluation Forum, 2000, p. 9). 

Effective impact assessment includes a 
differentiation between outcome indicators and program 
outcomes. Outcome indicators are specific performance 
elements that are targeted, measured, and tracked; 
program outcomes are the demonstrated differences the 
program has made, measured by the indicators. 
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The Practitioner’s Role in Assessment 

Program evaluations are only as valid as the metrics 
upon which they are based. Unreliable outcome 
indicators produce flawed program evaluations. 
Demonstrable and accurate assessment measures can 
guide the design of learning activities that align 
measures of success with intended learning objectives. 
Keen practitioners develop a proficiency in multi¬ 
faceted assessment that is grounded in theory and spans 
the learning spectrum at various levels. A widely 
accepted measurement of multi-dimensional learning is 
that of Bloom’s et.al Taxonomy of Learning Domains 
(see Appendix A). The taxonomy clearly identifies and 
delineates between levels of learning in each of the 
cognitive, psychomotor, and affective domains. To 
ensure outcome indicators are accurate for program 
evaluations, instructors must be proficient at not only 
recognizing learning at each of these levels, but also at 
designing corresponding objectives, activities, and 
metrics by which to assess attainment of the learning 
objectives. 

12-Step Outcomes-Based 
Program Evaluation Model 

Using the 12-Step Outcomes-Based Program 
Evaluation Model (see Appendix B), instructors, 
program administrators, and advisory committees can 
all take an active role in designing and conducting 
outcomes-based program evaluations that include 
specific and accurate metrics highly valued by program 
funders and accrediting bodies. 

Step 1 

Articulate a program overview. What is the overarching 
purpose of the program? Why does the program exist? 
What will participants be able to do after successful 
completion of the program? What difference(s) will the 
program make? What will be the measurable impacts of 
the program to the stakeholders? Design an evaluation 
plan. What are the expected outcomes of the program? 


What (if any) are the current standards by which the 
program will be measured? How/when will outcomes 
be measured? What are your specific outcome 
indicators? 

Step 2 

Outline the content of the program and determine the 
number of courses required to cover the program 
content. Identify the content for each course. Include the 
overarching Knowledge, Skills, and Attitudes (KSAs) 
that completers will be expected to demonstrate. 

Step 3 

Divide each course into units. List all the appropriate 
and relevant KSAs for each unit. 

Step 4 

Using Bloom’s Taxonomy, determine the targeted level 
within each learning domain for each KS A in each unit. 

Step 5 

Once the level of Domain has been targeted for each 
KSA, write an appropriate learning objective for each 
KSA. 

Step 6 

Design an appropriate learning activity for each KSA. 
Step 7 

Design an appropriate evaluation metric for each KSA. 
Step 8 

Deliver lessons and administer evaluations. 
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Step 9 

Analyze evaluation data such as class averages, 
quantitative data (production, performance time), and 
qualitative data (observations, perceptions, and/or 
narrative comments). Note that although data are 
collected on individuals, program results (not individual 
results) will be reported. Look for emerging trends 
during the data analysis process. 

Step 10 

Compare aggregate data to expected or to previous 
outcomes such as credentialing standards or 
benchmarks. 

Step 11 

Make recommendations for improvements at the unit, 
course, and program levels. 

Step 12 

Write a summative report for the Outcomes-Based 
portion of the Program Evaluation, addressing the 
elements listed in Step 1 of the Model. To what extent 
did the program fulfill its purpose? To what extent, and 
at what level, are the completers performing the 
anticipated KSAs? What difference(s) has the program 
made to the stakeholders? 

Conclusion 

The main function of outcomes-based evaluation is 
to improve the quality of a program by comparing the 
results with the intended program objectives. Effective 
assessment involves a continuous, ongoing process to 
identify successful aspects of the program and to 
discover the elements that need improvement and 
further development. These evaluations provide 
information for decisions concerning future 


programming. Outcome indicators, provided by 
instructors proficient in multi-leveled learning 
assessments, provide solid, valid, and reliable data for 
accurate outcomes-based program evaluations. 
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Appendix A 


A line of inquiry (Bloom, et al.) has resulted in the following identification of levels and activities in the 
cognitive, psychomotor, and affective domains of learning. 

Cognitive Domain 

The cognitive domain involves knowledge and the development of intellectual skills. This includes the recall or 
recognition of specific facts, procedural patterns, and concepts that serve in the development of intellectual abilities 
and skills. Based on Bloom’s model, originated in 1956, Anderson and Krathwohl (2001) propose the following 
revised levels of cognition: 

Remembering: Retrieving relevant knowledge from long-term memory 

Understanding: Determining the meaning of instructional messages, including oral, written, and graphic 
communication 

Applying: Carrying out or using a procedure in a given situation 

Analyzing: Breaking material into its constituent parts and detecting how the parts relate to one another and to 
an overall structure or purpose 

Evaluation: Making judgments based on criteria and standards 

Creating: Putting elements together to form a novel, coherent whole or make an original product 


Bloom's Taxonomy Updated 


Old Version New Version 
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Psychomotor Domain 

The psychomotor domain includes physical movement, coordination, and use of the motor-skill areas. Development 
of these skills requires practice and is measured in terms of speed, precision, distance, procedures, or techniques 
in execution. The levels, according to Dave, 1975, include: 

Imitation: Observing and patterning behavior after someone else 

Manipulation: Being able to perform certain actions by following instructions and practicing 

Precision: Refining, becoming more exact. Few errors are apparent. 

Articulation: Coordinating a series of actions, achieving harmony and internal consistency 

Naturalization: Having high level performance become natural, without needing to think much 

Affective Domain 

This domain includes the manner in which we deal with things emotionally, such as feelings, values, appreciation, 
enthusiasms, motivations, and attitudes. Krathwohl, Bloom, and Masia (1973) have defined the levels as: 

Receiving Phenomena: Awareness, willingness to hear, selected attention. 

Responding to Phenomena: Active participation on the part of the learners. Attends and reacts to a particular 
phenomenon. Learning outcomes may emphasize compliance in responding, willingness to respond, or 
satisfaction in responding (motivation). 

Valuing: The worth or value a person attaches to a particular object, phenomenon, or behavior. This ranges from 
simple acceptance to the more complex state of commitment. Valuing is based on the internalization of a 
set of specified values, while clues to these values are expressed in the learner's overt behavior and are often 
identifiable. 

Organization: Organizes values into priorities by contrasting different values, resolving conflicts between them, 
and creating a unique value system. The emphasis is on comparing, relating, and synthesizing values. 

Internalizing values (characterization): Has a value system that controls behavior. The behavior is pervasive, 
consistent, predictable, and most importantly, characteristic of the learner. Instructional objectives are 
concerned with the student's general patterns of adjustment (personal, social, emotional). 
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